Improve error message for invalid input in `Get()`s #11081

Eric-Arellano · 2020-10-29T22:11:14Z

Before:

Exception: WithDeps(Inner(InnerEntry { params: {Specs, Console}, rule: Task(Task { product: List, can_modify_workunit: false, clause: [Addresses, ListSubsystem, Console], gets: [Get { output: UnexpandedTargets, input: Addresses }], func: pants.backend.project_info.list_targets:64:list_targets(), cacheable: false, display_info: DisplayInfo { name: "pants.backend.project_info.list_targets.list_targets", desc: Some("list goal"), level: Debug } }) })) did not declare a dependency on JustGet(Get { output: UnexpandedTargets, input: int })

After:

Engine traceback:
  in select
  in pants.backend.project_info.list_targets.list_targets
Traceback (most recent call last):
  File "/Users/eric/DocsLocal/code/projects/pants/src/python/pants/engine/internals/native.py", line 65, in generator_send
    res = func.send(arg)
  File "/Users/eric/DocsLocal/code/projects/pants/src/python/pants/backend/project_info/list_targets.py", line 79, in list_targets
    targets = await Get(UnexpandedTargets, Addresses, 123)
  File "/Users/eric/DocsLocal/code/projects/pants/src/python/pants/util/meta.py", line 182, in new_init
    prev_init(self, *args, **kwargs)
  File "/Users/eric/DocsLocal/code/projects/pants/src/python/pants/engine/internals/selectors.py", line 159, in __init__
    self.input = self._validate_input(input_arg1, shorthand_form=False)
  File "/Users/eric/DocsLocal/code/projects/pants/src/python/pants/engine/internals/selectors.py", line 201, in _validate_input
    f"Invalid Get. The third argument `{input_}` must have the exact same type as the "
TypeError: Invalid Get. The third argument `123` must have the exact same type as the second argument, pants.engine.addresses.Addresses, but had the type <class 'int'>.

Note that the stacktrace includes the offending Get, including the line and file where it's set. This is because we now eagerly validate in the Python constructor for Get.

We can only apply this new error message for non-union Get calls; the engine must handle type errors relating to unions because Python does not have knowledge of UnionMembership in the Get constructor.

Closes #8349.

Not sure why we had this in the first place # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

# Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

coveralls · 2020-10-29T22:58:45Z

Coverage remained the same at 0.0% when pulling 7447d81 on Eric-Arellano:get-error into 47b583e on pantsbuild:master.

rcuza · 2020-10-29T23:06:04Z

Feedback on the After Error Message

The "After" error message is clearer.

There is still some ambiguity about which input is at fault. Is UnexpectedTargets supposed to be a single object of type Addresses or the Addresses part? I guess it is "obvious" that it is the Addresses part, but just the fact that I had to think about this for a few cycles makes me think it could be clearer still.

Is it true that Get(A, B)expects A to be of type UnexpandedTargets and B to be of type Addresses? If yes, then maybe:

Exception: Invalid input to `Get(A, B)`. Expected `B` to be of type `Addresses`, but got `123` (type `int`) instead.

I am not sure what I am supposed to do with See the rule list_targets() from pants.backend.project_info.list_targets (line 64). I assume that I need to go to the pantsbuild/pants repo and look at the code. More helpful would be the line number of the offending Get(A, B) but maybe that information isn't available to the function throwing the exception.

rcuza · 2020-10-29T23:09:31Z

re: pants.backend.project_info.list_targets (line 64): if we are going to send people to a goal to get more info, it at least needs a docstring.

I see that line 64 is dynamically determined which is très cool.

Eric-Arellano · 2020-10-29T23:13:55Z

I am not sure what I am supposed to do with See the rule list_targets() from pants.backend.project_info.list_targets (line 64).

This is the @rule that contains the offending Get(). That specific one is only an example. It's because we have the line number of the @rule where the Get happens, but not the line of the Get() itself.

--

But, in a perfect world, I think the error message could look something like this:

Exception: Invalid input to Get(A, B) on line 64 of project.my_module.foo. Expected the Get's input to be of type B, but got 123 (type int).

I'll see what we can do.

Eric-Arellano · 2020-10-29T23:32:22Z

I'll see what we can do.

Yeah I'm not finding a way in Python to automatically what line a constructor call happens at, that is where Get(A, B, b) is defined.

rcuza · 2020-10-30T00:25:10Z

I am not sure what I am supposed to do with See the rule list_targets() from pants.backend.project_info.list_targets (line 64).

This is the @rule that contains the offending Get(). That specific one is only an example. It's because we have the line number of the @rule where the Get happens, but not the line of the Get() itself.

I thought I might be reading it out of context (i.e. that the module was in the code throwing the error), but after I hit comment.

--

But, in a perfect world, I think the error message could look something like this:

Get(A, B) on line 64 of project.my_module.foo. Expected the Get's input to be of type B, but got 123 (type int).

If you can't get the line and just the module you can say, "Exception: Invalid input to Get(A, B) in project.my_module.foo (line 64). Expected the Get's input to be of type B, but got 123 (type int)."

stuhood · 2020-10-30T01:34:47Z

src/rust/engine/src/nodes.rs

-                      ty
-                    )),
-                  }
+            edges.entry_for(&dependency_key).cloned().ok_or_else(|| {


This has gotten pretty large, and would be good to extract into a helper function.

I plan to simplify the union code in a followup.

src/rust/engine/src/nodes.rs

# Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Eric-Arellano · 2020-10-30T04:10:37Z

PTAL. The error message was moved from Rust into the Python constructor. PR description is updated.

And rewrite part of it to use RuleRunner and to have better proximity. # Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Eric-Arellano · 2020-10-30T17:08:17Z

src/python/pants/engine/internals/selectors_test.py

+def test_invalid_get() -> None:
+    # Bad output type.
+    assert_invalid_get(
+        lambda: Get(1, str, "bob"),  # type: ignore[call-overload, no-any-return]
+        expected=(
+            "Invalid Get. The first argument (the output type) must be a type, but given "
+            f"`1` with type {int}."
+        ),
+    )


While these tests are brittle, I argue that's a feature. We know we need better error messages for the engine - this brittleness ensures we don't accidentally revert on the progress made in this PR.

Of course, we can rewrite the test if we want to reword the message.

Eric-Arellano · 2020-10-30T17:09:02Z

src/rust/engine/src/nodes.rs

-                      ty
-                    )),
-                  }
+            edges.entry_for(&dependency_key).cloned().ok_or_else(|| {


I plan to simplify the union code in a followup.

stuhood

Looks great! Thank you.

# Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Eric-Arellano added 2 commits October 29, 2020 13:23

Remove unnecessary Option

cc373c8

Not sure why we had this in the first place # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Improve error message for invalid input in Get()s

4aece80

# Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Eric-Arellano requested review from stuhood, tdyas and gshuflin October 29, 2020 22:11

gshuflin approved these changes Oct 30, 2020

View reviewed changes

stuhood approved these changes Oct 30, 2020

View reviewed changes

Eric-Arellano requested review from stuhood and gshuflin October 30, 2020 04:09

Move the error message to Python

71deaa8

# Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Fix scheduler_test.py

23abd07

And rewrite part of it to use RuleRunner and to have better proximity. # Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Eric-Arellano commented Oct 30, 2020

View reviewed changes

stuhood approved these changes Oct 30, 2020

View reviewed changes

Fix test string match failing

7447d81

# Rust tests and lints will be skipped. Delete if not intended. [ci skip-rust] # Building wheels and fs_util will be skipped. Delete if not intended. [ci skip-build-wheels]

Eric-Arellano merged commit 6d3f301 into pantsbuild:master Oct 30, 2020

Eric-Arellano deleted the get-error branch October 30, 2020 20:58

Eric-Arellano mentioned this pull request Nov 30, 2020

Rule parsing errors are terrifying. #10293

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Improve error message for invalid input in `Get()`s #11081

Improve error message for invalid input in `Get()`s #11081

Eric-Arellano commented Oct 29, 2020 •

edited

Loading

coveralls commented Oct 29, 2020 •

edited

Loading

rcuza commented Oct 29, 2020

rcuza commented Oct 29, 2020 •

edited

Loading

Eric-Arellano commented Oct 29, 2020

Eric-Arellano commented Oct 29, 2020

rcuza commented Oct 30, 2020

stuhood Oct 30, 2020

Eric-Arellano Oct 30, 2020

Eric-Arellano commented Oct 30, 2020

Eric-Arellano Oct 30, 2020

Eric-Arellano Oct 30, 2020

stuhood left a comment

Improve error message for invalid input in Get()s #11081

Improve error message for invalid input in Get()s #11081

Conversation

Eric-Arellano commented Oct 29, 2020 • edited Loading

coveralls commented Oct 29, 2020 • edited Loading

rcuza commented Oct 29, 2020

Feedback on the After Error Message

rcuza commented Oct 29, 2020 • edited Loading

Eric-Arellano commented Oct 29, 2020

Eric-Arellano commented Oct 29, 2020

rcuza commented Oct 30, 2020

stuhood Oct 30, 2020

Choose a reason for hiding this comment

Eric-Arellano Oct 30, 2020

Choose a reason for hiding this comment

Eric-Arellano commented Oct 30, 2020

Eric-Arellano Oct 30, 2020

Choose a reason for hiding this comment

Eric-Arellano Oct 30, 2020

Choose a reason for hiding this comment

stuhood left a comment

Choose a reason for hiding this comment

Improve error message for invalid input in `Get()`s #11081

Improve error message for invalid input in `Get()`s #11081

Eric-Arellano commented Oct 29, 2020 •

edited

Loading

coveralls commented Oct 29, 2020 •

edited

Loading

rcuza commented Oct 29, 2020 •

edited

Loading